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(57) Abstract 

Designs and operational methods to 
increase the dynamic range of image sen- 
sors (200) and APS devices (300) in par- 
ticular by achieving more than one inte- 
gration times for each pixel (302) thereof. 
An APS system (200) with more than one 
column-parallel signal chains (204; 206) 
for readout are described for maintaining 
a high frame rate in readout. Each active 
pixel (302) is sampled for multiple times 
during a single frame readout, thus result- 
ing in multiple integration times. The op- 
eration methods can also. be used to ob- 
tain multiple integration times for each 
pixel (302) with an APS design having a 
single column-parallel signal chain (104) 
for readout Furthermore; analog-to-digttal 
conversion of high speed and higji resolu- 
tion can be implemented! 
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AN IMAGE SENSOR WITH HIGH DYNAMIC RANGE LINEAR OUTPUT 



Origin Qf the Invention 
The invention described herein was made in the 
5 performance of work under a NASA contract and is 

subject to the provisions of Public Law 96-517(35 USC 
202) in which the Contractor has elected to retain 
title . 

Field ot the Invention 

10 The present invention relates to semiconductor 

image sensor, and more specifically, to a system and 
method of extending dynamic range in CMOS active pixel 
sensor circuits. 

15 Backgrou nd and Summary of the Invention 

An active pixel sensor ( "APS" ) is a special kind 

of light sensing device. Each active pixel includes a 

light sensing element and one or more active 

transistors within the pixel itself. The active 

20 transistors amplify and buffer the signals generated by 

the light sensing elements in the pixels. One type of 
such APS devices is disclosed in U.S. Patent No. 
5,471,515 by Fossum et al., the disclosure of which is 
incorporated herein by reference. 

25 APS devices represent an emerging technology in a 

wide range of imaging applications. APS has a number 
of significant advantages in comparison with the well- 
developed and widely used charge coupled devices 
(CCDs) ♦ Because of ~ the number of stages in a CCD, a 

30 nearly perfect charge transfer efficiency must be 

maintained in a CCD in order to maintain good signal 
fidelity. This usually makes CCDs susceptible to 
damage from both ionizing and displacement damage. In 
particular, the displacement damage caused by high- 

35 energy particles and photons may prove to be deadly to 

CCDs. 
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Large CCD arrays present engineering difficulties 
in manufacturing. Device miniaturization is difficult 
with CCD devices since the extreme requirements of CCDs 
necessitate a special formation process. This 
formation process prevents CCDs from being easily 
integrated with on-chip integrated circuits such as 
complementary-metal -oxide-semiconductor (CMOS) 
integrated circuits. CCDs also suffers limited 
spectral responsivity range and readout rates. 
Furthermore, the readout of CCDs is destructive, i.e., 
their stored value is destroyed upon reading out. 
Implementation of the nondestructive readout in a CCD 
device is difficult. 

In contrast, an APS device receives and processes 
input signals with the active pixel itself, thus 
eliminating the charge transfer over distances that are 
inherent in CCDs. Consequently, many drawbacks 
associated with CCDs are avoided in APS devices. For 
example, the performance of APS devices can be 
maintained as the array size increases. The APS 
readout rate is usually higher than that of CCDs. Since 
CMOS circuitry is often associated with the image 
sensor, the power consumption can be significantly 
reduced. APS devices are inherently compatible with 
CMOS processes, allowing reduced cost of manufacturing. 
Many on-chip operations and controls can be relatively 
easily implemented including timing and analog-to- 
digital conversion. APS devices are also less 
vulnerable to radiation damage and can be designed for 
non-destructive readout. Moreover, the active pixels 
of APS devices allow random access and on-chip signal 
processing. 

One important benchmark in performance of imaging 
devices is the ratio of the saturation level of the 
detectors and the noise level thereof or the signal-to- 
noise ratio (SNR) . This can be expressed in terms of 
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dynamic range of the device. The dynamic range is 
usually expressed in d-B by 201og{SNR) or in binary 
(bits) by log.(SNR). The larger the dynamic range, the 
better an imaging devices. 
5 In particular, a large dynamic range is desirable 

in applications for sensing low light signals and 
capturing images with large variations in brightness. 

The dynamic range of previously-reported CMOS- 
based APS circuits has been limited by both the 

10 saturation level of the signal chain circuit , which is 

typically about 1.2 volts for a 5-volt power supply, 
and the noise floor of the sensor, which is typically 
about 150jxV. This results in a dynamic range of 
approximately 78dB (13 bits), which is comparable to 

15 the dynamic range of the state-of-art CCD devices. 

The output voltage signal of an imaging device is 
dependent on the input light level, the efficiency of 
the optical coupling device and the detector 
characteristics including the quantum efficiency, the 

20 effective active sensing area, the integration time, 

and the electrical conversion gain of volts/electron. 
The output signal can be approximately expressed as the 
following: 



V = 



out = ~~Z T optics A dez T int I)Gf (D 



where <E> is the incident photon flux, f is the f-stop of 
25 the coupling optical system, T optlCJ is the transmission of 

the optical system, A d , t is the pixel size, T tnt is the. 
integration time, r| is the pixel quantum efficiency, 
and G is the conversion gain in volts/electron. The 
typical values for a APS device are f/8, 80% for T optlcs , 

30 20nm for A dec , 33ms for T int , 25% in T|, and G=10*iV/e\ 

respectively . 
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The exposure of an imaging device for a given 
hardware configuration is usually controlled by 
changing either the aperture (i.e., f-stop) or the 
integration time. If the input light level is low, the 
integration time is increased to improve the . signal-to- 
noise ratio. If the input light is bright, the 
integration time is reduced to avoid saturation of the 
detector while maintaining a high SNR. The lower limit 
of the integration time is set by the readout time. 

A number of prior-art techniques exist for 
controlling the integration time. For example, the 
integration time of a CMOS APS device with N rows of 
pixels can be controlled by resetting a row of pixels 
in advance of readout. If the readout time for one row 
of pixels is T roW the total frame readout time is NT rou . 
Since the pixels of a row are reset for a new 
integration upon readout, the integration time is 
simply the frame time NT row . 

FIG. 1 illustrates an exemplary APS. A pixel 
array 102 has N rows of pixels with each row having M 
columns. A column-parallel signal chain 194 such as a 
sampling capacitor bank with M storage cells is used 
for readout. 

In a specific operation of readout, a particular 
row is selected for readout at one time. The sensor 
data from the M pixels in the selected row is copied 
onto the capacitor bank 104. The copy process also 
resets the pixels in the selected row and begins a new 
integration. The M storage cells in the capacitor bank 
104 is then scanned sequentially for readout. The 
above readout process is then repeated for the next 
row. Therefore, the integration time for each pixel is 
identical and is equal to the readout time of a frame, 
i.e., NT cow . 

5 Another prior-art approach uses an electronic 

switching mechanism built in the pixel design of a CMOS 
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APS device, allowing electronic shuttering and 
simultaneous integration. This was disclosed in U. S. 
Provisional Application No. 60/010,305 filed on January 
22, 1996 by Fossum et al. 

The inventors of the present invention recognized 
a limitation of the above two methods in that the 
exposure remains the same for the entire image. This 
can result in distortion of the detected images in some 
circumstances. For example, an outdoor scene might be 
optimally exposed for the sunlit areas but overexposed 
in the shadowed areas. This can result in loss of 
detail in shadowed areas. 

One solution to the above problem is to use a 
nonlinear output sensor. Such a sensor is designed to 
have a high differential gain" for light levels, and low 
differential gain for high light levels. It is 
desirable that the nonlinear gain be achieved within 
each pixel. As a result, pixel to pixel variations in 
the transfer function can lead to unacceptable values 
of fixed pattern noise. 

Another approach is to implement multiple storage 
sites in each pixel in CCD devices to record different 
integration signals from the same pixel. In case of 
two storage sites in each pixel, one storage site 
corresponds to a short integration period and the 
second to a long integration period, thus increasing 
the dynamic range. The readout data can be transformed 
to a lower bit count encoding using nonlinear 
techniques for display, storage, and transmission. One 
limitation of this technique is a decreased readout 
rate since the readout time may be doubled since twice 
as much of the amount of data may need to be read. 
Another limitation is the reduced fill factor since two 
storage sites occupy considerable pixel area. 

In view of the above limitations, the present 
invention, describes a new technique in APS sensors and 
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operation methods thereof to increase their dynamic 
ranges. According to the preferred embodiments of the 
' present invention, at least two different integration 
times are obtained for each active pixel. One or more 
5 column-parallel signal chains are used to implement the 

multiple integration times for each active pixel. For 
an APS device with only one column-parallel signal 
chain for readout, each pixel is sampled multiple times 
during a single frame readout, thus resulting in 
10 multiple integration times. 

Another aspect of the present invention increases 
the frame rate by using more than one column-parallel 
signal chain to achieve multiple data points of 
different integration times for each pixel. A maximal 
15 frame rate is achieved for a given APS device by having 

the number of column-parallel signal chains match the 
number of different integration times. A preferred 
operation mode of such system in accordance with the 
present invention includes simultaneously copying 
20 multiple columns into the column-parallel signal chains 

and subsequently reading out all the column-parallel 
signal chains simultaneously with each being scanned rn 
a sequential manner. The frame rate is increased by a 
factor equal to the number of the column-parallel 
25 - signal chains as compared with achieving the same 
number of integration times with a single column- 
parallel signal chain for readout. The dynamic range 
of the APS device is thus extended by a factor given by 
the ratio of the longest integration time versus the 
30 shortest integration time. 

A new readout method is also described. This 
method can be used to improve the dynamic range of a 
conventional APS device having a single column-parallel 
signal chain by achieving multiple integration times 
35 for each active pixel. Special advantages are obtained 
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when used with non-destructive readout of an APS device 
in achieving multiple integration times of each pixel. 

Another aspect incorporates digital processors 
including analog- to-digital converters in the column- 
5 parallel signal chains. 

On-chip buffer memory units and/or multiple source 
followers may be used for each pixel to further enhance 
the flexibility and performance of the systems in 
accordance with the present invention. 
10 Brief De scription of the Drawings 

These and other advantages of the present 
invention will become more apparent in the light of the 
following detailed description of preferred embodiments 
thereof, as illustrated in the- accompanying drawings, 
15 in which: 

FIG. 1 illustrates an active pixel sensor with a 
single column-parallel signal chain to produce the same 
integration time for every pixel. 

FIG. 2 shows an active pixel sensor with two 
20 column-parallel signal chains for readout in accordance 

with a preferred embodiment of the present invention. 

FIG. 3 shows one example of an active pixel 
structure 

and a cell for the column-parallel signal chain 
25 disclosed by the U.S. Patent No. 5,471,515. 

FIG. 4 shows a readout method for achieving 
multiple integration times using an active pixel sensor 
with a single column-parallel signal chain. 

FIG. 5 shows another floor plan for positioning 
30 the two column-parallel signal chains relative to the 

active pixel array. 

npsnription of the Pref erred Embodiments 
The description of the preferred embodiments will 
35 use CMOS APS devices, as an example for implementation. 

However, it should be understood that the method and 
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design can also be implemented in other. active pixel 
sensors, and may be usable in other image acquiring 
systems. 



Design Architectures and Operation Modes 

FIG. 2 illustrates the first preferred embodiment 
200 of the present invention. An active pixel array 
202 has N rows and M columns, hence with a total of NxM 
pixels. Each pixel comprises a light sensing element 
and one or more active transistors. The active 
transistors amplify and buffer the signals generated by 
the light sensing element in the pixel. Each active 
pixel has one output source follower that is connected 
to the corresponding column bus.' There are two column- 
parallel signal chains for readout. Each of the signal 
chains, 204 and 206, has M storage cells for 
temporarily storing the electrical signals from the 
pixels being read. The column-parallel signal chains 
204 and 206 can be implemented with capacitor banks for. 
analog processing and analog- to-digital converters 
(ADCs) for digital processing or any other device that 
is capable of storing image information. 

The device 200 also includes an control circuit 
(not shown) having row and column decoders for 
addressing the proper row/column of pixels, clock 
generator circuits for synchronization, and readout 
circuits. The control circuit is operable to control 
the readout of the pixels and the operation of the 
column-parallel signal chain (s) in a desired sequence 
with desired integration time(s). 

An example of such active pixel and the storage 
cell in a column-parallel signal chain is shown in FIG. 
3. This is disclosed in the incorporated reference 
U.S. Patent Application No. 5,4 71,151 which describes 
an active pixel array with a single column-parallel 
signal chain for readout. FIG. 3 shows the active 
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pixel 302 including a phqtogate 310, a DC biased 
transfer gate 311, an output floating diffusion 314, a 
reset transistor 315, a drain diffusion 316, a gate 
source follower 317, and a row selection transistor 
5 318. The circuit 304 represents a storage cell shared 

by a column of active pixels. The storage cell 304 
includes a load transistor 320, a first output circuit 
321 for buffering and exporting the reset level, and a 
second output circuit 322 for buffering and exporting 

10 the signal level." In the architecture shown in FIG. 2, 

each active pixel of a column is connected to two such 
storage cells substantially similar to 304. It should 
be understood that other design of the active pixel and 
column storage cell can also be used in accordance with 

15 the present invention. 

In operation, a row of pixels in the active pixel 
array 202, e.g., the row n 210, is first selected for 
readout. Th'e signals representing the M pixels of the 
row n 210 are copied simultaneously to the 

20 corresponding M storage cells in the first column- 

parallel signal chain 201. The time duration of the 
copying process is T copy , typically about l-10p.s for an 
APS device. The copying process also resets the pixels 
in the row n 210 and a new integration in the row n 210 

25 is started. Secondly, the signals of M pixels in 

another row 220 that is shifted by A rows relative to 
row 210, i.e., the row (n-A) , are simultaneously, copied 
to the corresponding M storage cells in the second 
column-parallel signal chain 206 during time T copy and 

30 the pixels in the row (n-A) 220 is subsequently reset 

for another integration. The M storage cells in each 
of the first column-parallel signal chain 204 and the 
second column-parallel signal chain 206 are 
simultaneously and respectively scanned for readout. 

35 The M storage cells in each column-parallel signal 
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' chain is sequentially scanned and the signals therein 
are read out. This completes the readout of the row n 
210 and row (n-A) 220: The time for scanning and 
reading one pixel is T $Cflft , which is typically between 
5 lOOns-lOjis. Then, the next two rows are selected for 

readout in the same fashion.^ Therefore, a single row 
readout process reads two separated rows of pixels. 

The above readout process repeats until every row 
in the active pixel- array 202 are read to complete a 
10 frame. This results in two sets of data from every 

active pixel in the array 202, one from the first 
column-parallel signal chain 204 and another one from 
the second column-parallel signal chain 206, 
respectively. The two sets of data have different 
15 integration times and the two data points for the same 

pixel are read out at different times by a time lag of 
the time for a row readout process multiplied by A. 
The time for a row readout is 

T ro. = 2r copy ^r scdn/ (2) 

comparing to (T copy +MT SCMn ) in a row readout in the prior- 
20 art system as in FIG . 1. The total readout time for a 

frame is given by: 



T fr*me = NT ro»* (3) 



In the prior-art system of FIG. 1, the integration 
time for each pixel is the same and is the readout time 
for a frame. However, the two sets of readout from the 
two column-parallel signal chains 204 and 206 in the 
system of FIG. 2 have different integration times. The 
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integration time for the signals copied into the first 

column-parallel signal chain 204 is given by: 

r lint = ( w-A)T row , (4) 

and the integration time for the signals copied to the 
second column-parallel signal chain 206 is given by 

T 2^* T roV (5) 



10 



As a result, the dynamic range D 2 of the active 
pixel sensor 200 is improved over the dynamic range D { 
of the system 100 by a factor of T Uftt /T 2int : 



T 

D 2 (dB) =log(— + D x ( dB) ; (6) 



D 2 (bits) =log 2 (— ii££-) + D i (bits) . (7) 

T 2int 

For example, when N=512 and A=2, then a 13-bit dynamic 
range is thus extended by 8 bits to 21 bits, or from 
78dB to 126dB. 

15 The detected image is reconstructed by reading out 

the bright portions of the image with the data of short 
integration time T 2xnt from the second column-parallel 
signal chain 206. Conversely, darker portions of the 
image are read out using the data of long integration 

20 time T lint from the first column-parallel signal chain 

204.^ Therefore, portions of a scene which appear 
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saturated by the long integration time T Unl can be 
managed by using the short integration time T 2lnt . 

The second embodiment of the present invention 
uses the readout technique for the system 200 of FIG. 2 
in the system 100 of FIG. 1 to extend the dynamic 
range. The readout sequence is illustrated in FIG. 4. 
The row n of pixels 110 in the active pixel array 102 
is first selected for readout. The signals in the M 
pixels of the row n 110 are then copied simultaneously 
to the corresponding M storage cells in the column- 
parallel signal chain 104. The copying process also 
resets the pixels in the row n 110 and begins a new 
integration in the row n 110. Subsequently, the M 
storage cells of the column-parallel signal chain 104 
15 is sequentially scanned and the signals therein are 

read out. Next, the signals of M pixels of another 
row(n-A) 402 are simultaneously copied to the 
corresponding M storage cells in the column-parallel 
signal chain 104. The row 4C2 is displaced from the 
row 110 by A number of rows. The pixels in the row (n- 
A) 402 are subsequently reset for another integration. 
Subsequently, the M storage cells of the column- 
parallel signal chain 104 is again sequentially scanned 
and the signals therein are read out. This completes 
one row readout process which takes a total time of 
(2T eopy +2MT, ctn )=2T ro „. 

The above process then repeats for row (n+1) and 
row (n-A+1) , etc. until all N rows of active pixels in 
the array 102 are processed. The total frame readout 
time is U(2T cepy +2MT Jctn )= 2NT_, which is twice as long as 
the frame time in the prior-art readout shown in FIG. 
1- However, two sets of data are obtained using the 
readout of FIG. 4, one set having a first integration 
time, T Ilnt = (N-A)2r rot . and another set having a second 
35 integration time, T Unl = A2T ro „. Two data points of 
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different integration times from the same pixel are 
read out at two different times. The time lag between 
reading out the two data points is A ( 2 T copy +2MT SC4n ) -2AT rot ,. 
The increase in the dynamic range is essentially 
5 the same as in the first embodiment using a sensor 

design 200 of two column-parallel signal chains 
although the integration times are different. The 
second embodiment has an additional advantage of being 
compatible with previously-fabricated CMOS APS sensors 
10 with single column-parallel signal chain but has a 

reduced frame rate for readout due to the increased 
readout time per frame. 

A third embodiment implements more than two 
integration times in an active pixel array. For a 
15 number of k integration times (k>l) for each active 

pixel, the respective integration times are T lint =k A x T rou , 
T 2int =k A 2 T tow , and T klnl =k (N-A 1 -A 2 ...-A,. I ) T row , if the 

architecture 100 of FIG. 1 is used in which there is 
only one column-parallel signal chain for readout. The 
20 respective readout time for a frame is T frw =NkT row . For 

example, N=585 and k=4, A, may be chosen to be 1, A : = 8, 
A 3 =64 so that T lint - lx 4T row , T 2lnt = 8x 4T row/ T 3inc = 64x 4T row , 
and T 2iBt = 512 x 4T row . 

In addition, k column-parallel signal chains can 
25 be implemented and used in a similar way as in the 

first embodiment 200. L rows of pixels are read out 
simultaneously and stored in the k column-parallel 
signal chains, respectively. The k column-parallel 
signal chains are subsequently scanned and read out 
30 simultaneously. Thus, the frame time is reduced by 

approximately a factor of k while achieving essentially 
similar dynamic range increase comparing to the use of 
a single column-parallel signal chain for k integration 
times. 
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Furthermore, an intermediate number of column- 
parallel signal chains between 1 and k can be used to 
produce a desired set of k integration times and a 
frame rate tailored for a specific application. At 
» least one of the column-parallel chains is used to 

obtain at least two sets of data with different 
integration times. 

The above embodiments contemplate using a 
destructive readout, wherein an active pixel is reset 
for another integration after the signal therein is 
copied and read out. One advantage of the destructive 
readout is that the on-chip correlated doubling 
sampling can be implemented in many active pixel 
sensors to suppress noise such as the fixed patten 
noise and the kTC noise. This is disclosed by Mendis 
et al. in "CMOS Active Pixel Image Sensor", IEEE 
Transactions On Electron Devices, Vol . 4 1 ( 3 ) , pp . 4 52-4 53 , 
March, 1994. A non-destructive readout can also be 
used with the above embodiments for an increased 
dynamic range and possibly increased speed. Each pixel 
continues to integrate the signal and the pixel value 
is copied and read out without resetting. One 
advantage of the non-destructive readout is that a 
fewer number of rows are needed than that in the 
destructive readout to achieve the same multiple 
integration periods. For example, non-destructive 
readout can be. used to achieve four different 
integration times T linl = lx 4T tow , T J>n = 8x 4T tmt T )1B = 64x 
4T row , and T 21nt = 512 x 4T ro- using one column-parallel 
signal chain with 512 rows of pixels rather than 585 
rows in the destructive readout. This can be done by 
choosing A,=l, A : =7, and A,= 56 in a 512-row active pixel 
array. 
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Device Implementation 

(1) Choice of Pixel The architectures of active 
pixel array for exrended dynamic range previously 
described can be implemented with a number of 

5 possibilities. The preferred pixel design is a 

photogate CMOS APS shown in FIG. 3. However, many 
pixel designs of active-type and passive-type can be 
used. Examples of such include photodiode CMOS APS, a 
pinned photodiode CMOS APS, the passive pixel 
10 photodiode sensor, the charge modulation device active 

pixel sensor, and bipolar transistor-based pixels. 

(2) Chip Layout One or more column-parallel 
signal chains can use different elements in the pixels 
thereof including capacitors and analog-to-digital 

15 converters (ADCs) . The physical floor plans of 

multiple column-parallel signal chains such as FIG. 2 
and single column-parallel signal chain as in FIG. 1 
can be varied as desired in practicing the present 
invention. For example, the second column-parallel 

20 signal chain 206 can be positioned with the first 

column-parallel signal chain 204 at the lower part of 
the active pixel array 202 as illustrated by FIG. 5. 

(3) Multipl e Ogtpyt g pef -Elj&gl The inventors 
contemplate that a plurality of output source followers 

25 can be included in each pixel rather than one output 

source follower as described hereabove (e.g. , FIG. 3). 
Each output source follower can be attached to a 
separate column bus- and each column bus in turn can be 
attached to its own column parallel signal chain. This 

30 approach may provide greater flexibility in choosing 

the* integration times. 

(4) Pixel Synchronization The above-described 
embodiments use multiple data points of different 
integration times from the same pixel which are read 

35 out at different times rather than simultaneously. If 

it is desired to synchronize the output of the two or 
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more data points from the same pixel, an on-chip buffer 
memory is needed. The buffer memory holds the data 
from a pixel until all data points from that pixel are 
read out so that all the data points can be sent out 
5 simultaneously. In the case of zhe first embodiment 

200 of FIG. 2, the buffer memory should be A rows in 
size and has a capacity of storing half of the data 
from an image in the worst case. If on-chip ADC is 
used, the output of the buffer memory can be the input 
10 to the ADC. 

A High Performance Analog-to-Digital Converter ("ADC") 

A high speed ADC with a high resolution can be 
made using the described techniques of the present 
15 invention. The multiple data points generated from 

each pixel can be converted from analog to digital with 
multiple ADCs with each data point being processed by 
one ADC. This can generate both a high resolution 
conversion by combining the bits from each data point 
and a fast conversion speed since the multiple ADCs 
operate in parallel. 

For example, the architecture shown in FIG. 1 with 
a readout of four data points per pixel can be used for 
such a ADC. For illustration purpose, it is assumed 
25 that N=585, k=4 , A, = l, A 2 =8, A,= 64 so that T UBl = lx 4T tw , 

T«„»- 8x 4T ro- , T 3int = 64x 4T tow , and T, lnl = 512 x 4T tow . If each 
data point is converted using a 3-bit ADC, an ADC word 
length of 12 bits, can be generated by patching the four 
outputs of the ADCs together. The time domain, which 
can be accurately controlled, is used to provide the 
effective gain of 8x' between each of the four data 
points. This is shown in Table 1. Each of the four 
ADCs is shown as a single slope ADC, with an increment 
of 0.250mV per step and a total of 8 levels. The time 
required for each conversion is 8 cycles. Since the 
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four ADCs can be operated in parallel, the total time 
to achieve the 12-bit resolution could remain as 8 
cycles. Typically, each cycle is of the order l^ts so 
that a complete conversion can be finished in 8jis . The 
bits from the four ADCs corresponding to four data 
points are reassembled externally to generate the 12- 
bit output. 
Table 1 













Integration 
Time (T w ) 


1 


8 


6.4 


512 


Number of Steps 


8 


8 


8 


8 


Actual volts 
per step 


0.250 mV 


0.250 mV 


0.250 mV 


0.250 mV 


Effective voles 
per step 


128.000 mV 


16.000 raV 


2.000 mV 


0.250 mV 


Full effective 
range 


1024.000 mV 


128.000 mV 


16.000 raV 


2.000 mV 



It is noted that achieving n-bit dynamic range is 
not equivalent to obtaining n-bit resolution. For 
example, the sensor of the above-described case could 
be saturated for the three long integration times for a 
given light intensity at a pixel, thus only the data 
with the shortest integration time is used. One way to 
improve the number of significant digits is to used a 
higher resolution ADC to have some overlap in 
conversion range. 

Although the present invention has been described 
in detail with reference to several embodiments, one 
ordinarily skilled in the art to which this invention 
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pertains will 'appreciate that various modifications and 
enhancements may be made in the described embodiments. 

For example, the techniques of the embodiments 
described use readout units of rows and column-parallel 
i readout chains. It should be appreciated that readout 

units could be other pixel blocks, such as neighboring 
pixel patches (e.g., 7x7 pixel areas), or any other 
desired shape. Accordingly, the readout chains can be 
changed to accommodate the choice of readout units. 
While the embodiments describe obtaining two 
integration signals for each pixel being stored, it 
should be understood that any number, e.g., 3 or 4, 
integration times ; could be used. Additional column- 
parallel signal chains may be added to maintain the 
15 frame rate while increasing the number of integration 

times. Alternatively, the chip design can be 
maintained at a price of a slower frame rate to 
increase the number of integration times using the 
readout method of the present invention. 

These and other modifications and enhancements are 
intended to be encompassed by the following claims. 
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What is claimed is: 

1 1. An imaging device, comprising: 

2 a detector array having a plurality of pixel 

3 circuits disposed relative to each other on a 

4 semiconductor substrate, each of said pixel circuits 

5 including an optical sensor to receive light and an 

6 electronic element to convert said light into an 

7 electrical signal, said pixel circuits operating to 

8 generate an indicia indicative of an input scene; 

9 a first signal buffer array having a 

10 plurality of buffer cells disposed relative to said 

11 detector array on said substrate, said buffer cells in 

12 said first signal buffer array respectively operating 

13 to store electrical signals from a first patch of said 

14 pixel circuits; 

15 a second signal buffer array having a 

16 plurality of buffer cells disposed relative to said 

17 detector array on said substrate, said buffer cells in 

18 said second signal buffer array respectively operating 

19 to store electrical signals from a second patch of said 

20 pixel circuits; and 

21 a controlling circuit controlling said first 

22 and second signal buffer arrays in a way such that each 

23 of said pixel circuits is sampled into both said first 

24 and second signal buffer arrays at least once during a 

25 readout of said detector array. 

1 2. A device as in claim 1, further comprising: 

2 a transfer gate and a first transistor source 

3 follower disposed on said substrate in said electronic 

4 element in each of said pixel circuits, electrically 

5 connected to said first and second signal buffer 

6 arrays, said transfer gate relaying a signal from said 

7 optical sensor to said first source fbllower; 

8 a semiconductor switch in each of said buffer 

9 cells in said first and said second signal buffer 
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arrays, electrically connecting said buffer cell to 
said first source follower in a patch of said pixel 
circuits; and 

said controlling circuit controlling said • 
switch of each of said buffer cells in said first and 
second signal buffer arrays to allow said electrical 
signals in said first and second signal buffer arrays 
having different integration times. 

3. A device as in claim 2, further comprising a 
buffer memory formed on said substrate operating to 
store a plurality of data of different integration 
times obtained from said pixel circuits at different 
times in a frame readout. 

4. A device as in claim 2, further comprising a 
plurality of analog- tb-digital converters in each of 
said buffer cells in said first and second signal 
buffer arrays, operating in parallel to convert 
electrical signals of different integration times in 
said buffers into digital format, thus forming a 
digital indicia indicative of said input scene. 

5. A device as in claim 2, wherein said optical 
sensor is a phototransistor or a photodiode. 

6. A device as in claim 2, wherein said controlling 
circuit has means for enabling each of said pixel 
circuits to accumulate signals produced by said optical 
sensor during consecutive readouts by one of said 
buffer cells in said first and second signal buffer 
arrays . 

7. A device as in claim 2, further comprising: 

a reset gate in each of said pixel circuits 
being disposed on said substrate; and 
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4 said controlling circuit controlling said 

5 reset gate to delete accumulated signals produced by 

6 said optical sensor in each of said pixel circuits upon 
a completion of a sampling by either said first signal 

8 buffer array or said second signal buffer array. 

1 8. A device as in claim 2, wherein said controlling 

2 circuit operates to sample each of said pixel circuits 

3 at least three times with said first and second signal 

4 buffer arrays during said readout of said detector 

5 array to generate three sets of data with different 

6 integration times for each of said pixel circuits, 

7 thereby at least one of said first and second signal 

8 buffer arrays sampling said pixel circuits twice. 

1 9. A device as in claim 2, further comprising a 

2 plurality of additional signal buffer arrays disposed 

3 relative to said detector array on said substrate, said 

4 additional signal buffer arrays being substantially 

5 similar to said first and second signal buffer arrays, 

6 said controlling circuit controlling said signal buffer 

7 arrays in a way such that each of said pixel circuits 

8 is respectively sampled into each of said signal buffer 

9 arrays at least once during a readout of said detector 
10 array. 

1 10. A device as in claim 2, further comprising a 

2 second transistor source follower disposed on said 

3 substrate relative to said first source follower in 

4 said electronic element in each of said pixel circuits, 

5 said second source follower being electrically parallel 

6 with respect to said first source follower, said 

7 control circuit controlling said first source follower 

8 to read out a first signal from said optical sensor 
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9 with a first integration time and said second source 

10 follower to read out a second signal from said optical 
n sensor with a second integration time, respectively. 

1 11. An imaging device, comprising: 

2 a detector array having a plurality of pixel 

3 circuits disposed relative to each other on a 

4 semiconductor substrate, each of said pixel circuits 

5 including an optical sensor to receive light and an 

6 electronic element to convert said light into an 

i electrical signal, said pixel circuits operating to 

generate an indicia indicative of an input scene; 

a signal buffer array having a plurality of 
buffer cells, each of said buffer cells having a 

11 semiconductor switch formed on said substrate and 

12 electrically connected to said electronic element of a 

13 patch of said pixel circuits to store electrical 

14 signals from a pixel circuit in said patch of said 

15 pixel circuits; and 

16 a controlling circuit operating said switch 
in each of said buffer cells in said signal buffer 
array in a way such that each of said pixel circuits is 

19 sampled into said signal buffer array at least twice 

20 with different integration times during a readout of 

21 said detector array. 

1 12, A device as in claim 11, further comprising: 

2 means in said controlling circuit for 

3 selecting a first patch of said pixel circuits in said 

4 detector array, said electronic element in each of said 

5 pixel circuits in said first patch and said switch in 

6 each of said buffer cells operating in combination to 

7 simultaneously read a first set of electrical signals 

8 in said first patch pixel-by-pixel into said signal 

9 buffer array; 
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10 said signal buffer array sequentially 

11 exporting said first set of electrical signals stored 

12 in said buffer cells according to a first operation of 

13 said controlling circuit, each of said buffer cells 

14 resetting to a null after said exporting; 

15 means in said controlling circuit for 

16 subsequently selecting a second patch of said pixel 

17 circuits, said electronic element in each of said pixel 

18 circuits in said second patch and said switch in each 

19 of said buffer cells operating in combination to 

20 simultaneously read a second set of electrical signals 

21 in said second patch pixel-by-pixel into said signal 

22 buffer array; and 

23 said signal buffer array sequentially 

24 exporting said second set of electrical signals stored 

25 in said buffer cells according to a second operation of 

26 said controlling circuit. 

1 13. A device as in claim 12, wherein said controlling 

2 circuit has means for enabling each of said pixel 

3 circuits to accumulate signals produced by said optical 

4 sensor during consecutive readouts by one of said 

5 buffer cells in said signal buffer array. 

1 14. A device as in claim 12, further comprising: 

2 a reset gate in each of said pixel circuits 

3 disposed on said substrate; and 

4 means in -said controlling circuit for 

5 controlling said reset gate to delete accumulated 

6 signals produced by said optical sensor in each of said 

7 pixel circuits upon a completion of a sampling by said 

8 signal buffer array. 
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1 15. A method of detecting an optical scene, 

2 comprising: 

3 providing a two-dimensional array of pixels, 

4 each of said pixels having an optical detector and an 

5 electronic pixel circuit for converting input light 

6 from said optical scene into an electrical signal; and 

7 sampling electrical signals in said pixels in 

8 a selected spatial and temporal sequence to obtain a 

9 plurality of pixel data with different integration 

10 times from each of said pixels during a readout of said 

11 array. 

1 16. A method as in claim 15, further including: 

2 selecting one data from said plurality of 

3 pixel data for each of said pixels according to a 

4 predetermined criterion to form an indicia indicative 

5 of said optical scene. 

1 17. A method as in claim 15, further comprising: 

2 partitioning said pixels in said array into a 

3 plurality of pixel patches, each of said pixel patches 

4 having equal number of said pixels; 

5 providing a first signal readout buffer array 

6 having buffer cells, said buffer cells respectively 

7 corresponding to said pixels in each of said pixel 
.8 patches; 

9 simultaneously reading a first pixel patch 

10 into said buffer cells, of . said, first readout buffer, 

11 array on one-to-one correspondence, thereby obtaining a 

12 first set of electrical signals that are characterized 

13 by a first integration time, in said readout buffer 

14 array; 

15 sequentially scanning said buffer cells in 

16 said first readout buffer array and reading out said 

17 first set of electrical signals; 
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13 selecting a second pixel patch in said array 

19 according to a selection criterion; 

20 then simultaneously reading said second pixel 

21 patch into said buffer cells of said first readout 

22 buffer array on one-to-one correspondence, thereby 

23 obtaining a second set of electrical signals that are 

24 characterized by a second integration time in said 

25 first readout buffex array; and 

26 then sequentially scanning said buffer cells 

27 in said first readout buffer array and reading out said 

28 second set of electrical signals. 

1 18. A method as in claim 17, wherein said second pixel 

2 patch is spatially separated from said first pixel 

3 patch by a predetermined number of pixel patches, 

4 thereby said. second integration time and said first 

5 integration time having a difference affected by said 

6 predetermined number of pixel patches, 

1 19. A method as in claim 15, further comprising: 

2 partitioning said pixels in said array into a 

3 plurality of pixel patches, each of said pixel patches 

4 having equal number of said pixels; 

5 providing a first signal readout buffer array 

6 having, buffer cells,., said buffer cells respectively 

7 corresponding to said pixels in each of said pixel 

8 patches; 

9 providing a second signal readout buffer 

10 array with multiple buffer cells respectively 

11 corresponding to said pixels in each of said pixel 

12 patches; 

13 reading a first pixel patch into said buffer 

14 cells in said first readout buffer array on one-to-one 

15 correspondence, thereby obtaining a first set of 

16 electrical signals that are characterized by a first 

17 integration time in said first readout buffer array; 
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^ 8 selecting a second pixel patch in said array 

19 according to a selection criterion; 

20 then simultaneously reading said second pixel 

21 patch into said buffer cells of said second readout 

22 buffer array on one-to-one. correspondence, thereby 

23 obtaining a second set of electrical signals that are 

24 characterized by a second integration time in said 

25 second readout buffer array; and 

26 then simultaneously scanning said- buffer 

27 cells in said first and second readout buffer arrays 
and reading out said first set of electrical signals 
and said second set of electrical signals, 

30 respectively. 

1 20. A method as in claim 15, further comprising: 

2 performing an analog-to-digital conversion on 

3 each of said plurality of pixel data with different 

4 integration times from said pixels to obtain a digital 

5 representation thereof; and 

6 forming an digital indicia indicative of said 

7 optical scene based on said digital representation. 

1 21. A method as in claim 15, further comprising: 

2 depleting electrical signal in said 

3 electronic pixel circuit upon completion of obtaining a 

4 pixel data from each of said pixels during said readout 

5 of said array, thereby a subsequent pixel data being an 

6 accumulative signal of a different integration process. 

1 22. A method as in claim 15, wherein a first pixel 

2 data with a first integration time of said plurality of 

3 pixel data for a pixel is a sum of an accumulative 

4 signal during a first duration and a second pixel data 
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5 with a second integration time that is obtained right 

6 before the beginning of said first duration, thereby 

7 said first integration time being a sum of said first 

8 duration and said second integration time. 



WO 97/1 7800 



1/5 



PCT7US96/18287 



102 



110 



r 



Row n 
Row n+1 



column-parallel 
signal chain 



104 



sequentially scanned 
for readout 



100 



FIG. 1 



WO 97/17800 



2/5 



PCT/US96/18287 



2nd column-parallel 
signal chain 



Row n-A 



Row n 



1st column-parallel 
signal chain 



sequentially scanned 
for readout 



206 



202 



220 



210 



204 



sequentially scanned 
for readout 



200 



FIG. 2 



WO 97/1 7800 



PCT/US96/I8287 



3/5 



32-2. 




32) 



WO 97/17800 



PCT/US96/18287 



4/5 



102 



il 



402 



Row n-A 



sequentially scanned 
for readout 



110 



Row n 



column-parallel 
signal chain 



104 



100 



FIG. 4 



WO 97/17800 



PCT/US96/18287 



5/5 



Imaging Pixel Array 



1st column-parallel 
signal chain 

2nd column-parallel 
signal chain 

































































































































' 


:-: : :*: ; Xv. 


































































































•r* 










*• 
















































































































I 




I 

























Row n-A 



Row n 



sequentially scanned 
for readout 



FIG. 5 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US967 18287 



A. CLASSIFICATION OF SUBJECT MATTER 
IPC(6) H04N 3/143; H04N. 5/335 

US CL : 348/302. 311,316. 319. 321. 221. 224.. 229; 230. 246. 297 
According to International Patent CUsiiGcation (IPC) or to both national classification and IPC 



B. 



FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 348/302. 311,316; 319. 321. 22i; 224, 229. 230. 246. 297 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and. where practicable, search terms used) 
APS, IEEE Abstracts 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



X 
Y 
Y 



X 
Y 



US, A, 4,819,070 (Hynecek) 04 APRIL 1989, col 11, lines 
20-55. 



IEEE Journal of Solid-State Circuits, "A NEW MOS IMAGER 
USING PHOTODIODE AS CURRENT SOURCE", VoL 2, No. 8, 
(Kyomasu), August 1991, Figure lb. 

US, A, 5,084,704 (Parrish) 28 JANUARY 1992, col. 2, lines 
14-20. 

US, A, 5,264,994 (Takemura) 23 NOVEMBER 1993, Figure 
3, col. 3, lines 24-27. 



15, 16, 17, 18 

1, 19, 20, 21 

2, 3, 5, 6, 7 

4, 20 
11 

8,12,13,14 



X Further documents arc lilted in the continuation of Bo* C. [ | See patent family annex. 



•o* 

-p. 



Special oale< one* of cited document: 

document deitnina the general ataac of the art which is not conaidered 
to be port of particular relevance 

earlier document published am or after the naernabonai filing date 

document which may throw doub* en priority caum(i) or 
cited to eetaolish the pubucabon data of an o th er citanon 
apecial reason (at specified) 

document ref«rrb< to aa oral disclosure, uae. exhibition or other 



bacr Harumcmi pubuabed after the 
date and not m conflict with the 
praacipke or theory underfyina the av 



relevance; the cU* 
cannot be considered to 
m taken atone 



document of particular relevance: the cleaned 
considered to involve an mvcnnVe step wben 
combined with ooc or more other much documents, 
t obvious to • peraoa skilled in (he art 



filing dale 
eked to 



or priority 



document published prior to (be i 



I fUinf date but bier than 



of t 



family 



Date of ihc actual completion of the international search 
18 FEBRUARY 1997 



Dale of mailing of the international search report 

07 MAR 1997 



Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box'PCT 

Washington. D.C 20231 
Facsimile No . (703)- 305-3230 



^Jyjoirutenscn 
Telephone No. (703) 305-9871 



Form PCT/IS A/210 (second sheettfJuly 1992)* 



INTERNATIONAL SEARCH REPORT 



Inlemaijonal application No. 
POT/US96/ 18287' 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document, with indication, where appropriate, of the relevant passage*. 



Relevant to claim No. 



US, A, 4,959,723 (HASHIMOTO) 25 SEPTEMBER 1990, 
col. 10, lines 10-24. 

US, A, 4,706,123 (CHAUTEMPS) 10 NOVEMBER 1987, 
Figures 1, 2c. 

US, A, 5,162,914 (TAKAHASHI ET AL.) 10 NOVEMBER 
1992, Figure 3. 



10, 12, 13, 14 



22 



Form PCT7ISA/2I0 (continuation of second iheet)(July 1992)* 



